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COLOR TO GRAYSCALE CONVERSION 
METHOD AND APPARATUS 

[0001] The presently disclosed method and apparatus relates to digital imaging and 
especially to a method for converting color images into grayscale images. 

[0002] A color image sent to a monochrome output device should undergo a color- 
to-grayscale transformation. Such a transform typically retains the luminance channel or a 
derivative thereof. A problem with this approach is that the distinction between two different 
colors of similar luminance is lost. This loss can be particularly objectionable if the two 
colors are spatially adjacent. 

[0003] Considerable effort has been spent in finding a better color to grayscale 
transformation. Many of these techniques have their own flaws. Mappings based only on 
color information suffer from a "many-to-one" mapping problem. Also, attempts to take the 
Helmholtz-Kohlrausch effect into account (saturated colors appear lighter) have shown no 
systematic improvement. Previous methods have also included converting color palettes to 
texture to preserve distinction among different colors. While texture-based approaches can be 
quite effective in preserving distinction for certain types of images (e.g., graphics with regions 
of constant color), their global nature may not be desirable in some cases. For example, 
introduction of texture into smooth fleshtone regions in a portrait most commonly produces an 
undesirable appearance. 

[0004] Other approaches map a fixed palette of colors to grayscale by preserving 
relative 3-D color differences along the 1-D luminance axis. While this is an effective 
technique for business graphics having few colors, it does not readily extend to the general 
case of pictorial imagery or complex graphics with smoothly varying sweeps. 
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[0005] The method proposed in this disclosure uses the same underlying principle of 
spatial color processing as the gamut mapping method disclosed in U.S. Patent No. 6,621,923, 
hereby incorporated by reference. In the '923 patent a method of spatial gamut mapping is 
described that preserves local luminance edge information lost from the gamut-mapping step, 
thereby making the actual color mapping a space variant function, which we refer to as 
"spatial mapping," indicating that color, as well as neighborhood information is used in the 
processing step. The disclosed method describes computing the difference between the 
original and gamut-mapped luminance image, processing through a high-pass filter, and 
adding this back to the gamut-mapped luminance. The embodiments disclosed herein are 
based on a similar idea of feeding information lost in the color to grayscale mapping back into 
the system. However, it pays attention to errors originating from the mapping of color edges 
rather than pure luminance attributes. For simplicity, one might derive this error as a function 
of chrominance edges, thereby separating the maintained luminance edge information. It 
should be understood that here and in the following we are using the term luminance and 
chrominance to indicate a general description based on a color space that has one component 
along an axis that might be considered a "good" black and white representation of the input 
data, and two axes along directions that are indicative of color. Examples of such spaces are 
Lab, YCrCb, Xerox YES and the like. 

[0006] To improve the quality of grayscale images created from a color image, at 
least some pixels of the color image having the same color are mapped to different grays in 
the grayscale image depending on the spatial surround of each pixel. 

[0007] The method disclosed herein locally preserves the distinction between 
adjacent colors (i.e., along color edges) by introducing high-frequency chrominance 
information into the luminance channel. High frequency chrominance information 
corresponds to color edges. In embodiments, a high-pass filter is used as an efficient edge 
indicator. Embodiments include applying a high-pass filter to the chrominance components 
of a color image and using the result to modify the luminance channel of the image. 
Embodiments also include weighting the output with a luminance-dependent term. Other 
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embodiments might use different linear or non-linear filters, including logical filters, as long 
as the filter output is indicative of edge location and magnitude. 

[0008] Embodiments also include a system for converting a color image to a 
grayscale image. The system includes an image input device that receives a color image 
having luminance and chrominance components, an edge detector including a high pass filter 
for computing high-pass filtered chrominance components from the received chrominance 
components, a feedback unit, which modifies the luminance component based upon the high- 
pass filtered chrominance components, and an output device, that receives the modified 
luminance component and outputs a grayscale image based upon the modified luminance 
component. 

[0009] Embodiments also include a grayscale image created from a color image, 
wherein at least some pixels of the color image having the same color are mapped to different 
grays in the grayscale image depending on the spatial surround of each pixel. 

[0010] Embodiments also include a method for improving a color to grayscale 
transformation of an image composed of a plurality of pixels. The method includes selecting 
a subset of the plurality of pixels based upon at least one predetermined criterion derived from 
a local spatial neighborhood of the plurality of pixels, adjusting the luminance components of 
each of the subset of the plurality of pixels based upon the chrominance information of the 
same plurality of pixels, generating an output image based upon the adjusted luminance 
component. In embodiments, the predetermined criterion includes only selecting pixels in 
close proximity to an edge. 

[0011] Various exemplary embodiments will be described in detail, with reference 
to the following figures, wherein: 

[0012] Figure 1 is a block diagram of a method for color to grayscale conversion. 

[0013] Figure 2 illustrates the interaction between luminance and chrominance 

edges. 

[0014] Figure 3 illustrates exemplary function f(L hp ) used to compute a chrominance 
weighting factor. 



3 



[00.15] Figure 4 is a flowchart of a method for converting color images to grayscale 

images. 

[0016] Figure 5 schematically illustrates a color to grayscale conversion system. 

[0017] The following description discloses a method of improving edges and details 
in grayscale versions of color images. 

[0018] Reference is now made to Figure 1. To use the method disclosed herein, the 
information from a color image needs to be in a luminance-chrominance representation, such 
as L*a*b*, YES, YCbCr, etc. If the image is in an RGB space, it should be first converted to 
such a luminance-chrominance representation for simplicity of edge calculation. The 
particular luminance-chrominance representation used should not matter. While the rest of 
the description uses a L*a*b* representation, the method should work with data in any 
luminance-chrominance representation. 

[0019] Converting color images to grayscale images typically involves passing the 
luminance component, L*, of the image directly to the output device with the necessary 
processing required to convert L* into a device dependent signal. In the case of a printer, for 
example, the latter is usually the black colorant. 

[0020] To improve the output, the first step of the algorithm computes high-pass 
filtered versions of all three channels. These will be denoted as L hp9 and b^. The high-pass 
filters are used to isolate the information regarding luminance and chrominance edges where 
the frequencies are going to be largest. The particular frequency cutoff and the particular 
filter used will vary based upon, for example, the input image, the processor being used, and 
the output device. 

[0021] In embodiments, the chrominance components of the image are then 
combined into a single signal that represents high-frequency chrominance information. 
One candidate is the Euclidean metric: 




(i) 
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In order to lessen computational time, an alternative combined chrominance component could 
be used, such as the computationally simpler 1-norm metric: 



^=KI+W ' (2) 

[0022] In embodiments, the signal is then multiplied by a weight factor w that is a 
function of the high-pass luminance signal L^. This weight factor, denoted w = g(L^, is 
chosen with two criteria in mind: (1) to use the chrominance information only to enhance 
edges, and not to weaken them, and (2) to introduce the chrominance variation only at 
locations where the luminance variation is not sufficient to distinguish the local image 
variation. To satisfy the aforementioned two criteria, the chosen weighting function is given 
by: 

w = g( L h P ) = sign(L hp ) x sign(c hp ) x /( L hp |) (3) 

wherein the sign-functions and the f-function deal with the first and second criteria, 
respectively. 

[0023] In embodiments, the first criterion is resolved using a "sign function," 
applied to L ¥ and c ¥ . In embodiments, sign(x) takes on a value of +1 when x > 0, and -1 
when x < 0. The sign function is applied to high-pass-filtered luminance and chrominance 
components at each pixel. Using such a function, the high-pass filtered chrominance 
information is used to modify the luminance information only so as to enhance the luminance 
variation. The polarity or sign of the high-pass chrominance correction is adjusted, via the 
sign functions, to match that of the high-pass luminance signal. Specifically, before the high- 
pass-filtered chrominance information is used to modify the luminance signal, the high-pass- 
filtered chrominance information is multiplied by both sign(L^ and sign(c^. This ensures 
that the high-pass-filtered chrominance information will have the same sign as the high-pass- 
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filtered luminance information at an edge and therefore will only enhance the edge, rather 
than reduce it 

[0024] By way of example, consider the scenario shown in Figure 2 where an edge 
in an image is made up of variations in both luminance 2(a) and chrominance 2(d), but in 
opposite polarities. (This is a very common occurrence; e.g., at an edge between white and a 
colorful region; the white side would be of greater luminance, but the colorful region would 
have a higher chrominance.) The high-pass luminance signal L^Jx) and its corresponding sign 
function in the vicinity of the edge are shown in Figures 2(b) and 2(c) respectively. The high- 
pass chrominance signal c hp and its corresponding sign function in the vicinity of the edge are 
shown in Figures 2(e) and 2(f) respectively. Adding the high-pass chrominance signal 2(e) 
directly to the luminance signal in Fig. 2(a) would actually smoothen rather than enhance the 
luminance signal. To avoid this, the sign of the high-pass luminance signal L hp (Fig. 2c) is 
multiplied by the sign of the high-pass chrominance signal (Fig. 2f). This results in the 
overall sign function being -1 everywhere in the vicinity of this edge. Multiplying the high- 
pass chrominance signal in Fig. 2(e) by the overall sign function thereby effectively inverts it 
so that its polarity matches that of the high-pass luminance signal of Fig. 2b. This inverted 
high-pass chrominance signal is then added to the original luminance signal in Fig. 2(a). 

[0025] A number of functions may be used to resolve the second criterion, i.e., 
introduce the chrominance variation only in "areas" where the luminance variation is not 
sufficient to distinguish the local image variation. To achieve this, the amount of 
chrominance feedback is reduced when the luminance variation is large. Reducing the 
amount of chrominance feedback when the luminance variation is large is a conservative 
strategy that avoids excessive edge enhancement in regions that show sufficient detail. Figure 
3 illustrates one of many embodiments of a function / that may be used. Parameters K, Bl, 
and B2 control the amount of luminance edge enhancement as a function of the strength of the 
luminance edge in the original color image. The individual parameters of the function will 
depend on, for example, the desired output, and the particular qualities of the output device. 
Moreover, many functions can be chosen that fulfill the desired characteristics of a 
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monotonically non-increasing function. The form of the /-function will be dependent on 
multiple factors, such as, for example, the printer or display device that will generate the 
grayscale image. However, function / should be solely dependent upon the magnitude of the 
high-pass-filtered luminance signal, denoted |LJ, and not its polarity. 

[0026] In embodiments, an optional processing of is also advantageously 
performed to reduce noise and noise sensitivity. In areas where the chrominance edge 
indication is small, i.e., is very small, setting to zero could eliminate the feedback. This 
would prevent chrominance noise from leaking into the luminance channel. 

[0027] During experiments, color images were used for evaluating the method 
outlined above. The color prints were generated on a color-characterized Xerox Phaser 7700 
printer. In both cases, the L* image was printed using only black toner on the Phaser 7700, 
simulating a grayscale device. For the chosen prints and printer, the parameters of the 
weighting function j[L hp ) were chosen as K-l, Bl = 15, and B2 = 40 and arrived at on an 
empirical basis. Larger values of the parameters K, Bl, B2 result in more aggressive 
enhancement of high frequencies. The high-pass filters for the chrominance error filtering 
were chosen based on the filters used in the 923 patent. For three pictorials, a simple 15x15 
high-pass filter was used. For the text image, a smaller 5x5 high-pass filter was found to be 
more effective. Custom tuning of the various parameters will likely give additional 
improvements and would be made as a function of the output device, customer preferences, 
etc. 

[0028] The embodiments discussed herein describe a method for enhancing edges of 
an image when the image is converted from color to grayscale. The method comprises 
adjusting a luminance component of a subset of the plurality of pixels constituting the image 
based upon the chrominance information of those same pixels and generating an output image 
based upon the adjusted luminance component. The subset of the plurality of pixels is 
proximate to an edge between one object and another and pixels not proximate to the edge are 
not adjusted. 
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[0029] Figure 4 illustrates a flow chart of an embodiment of the method disclosed 
herein. The exact order of the steps outlined in this flowchart is not required. Additionally, 
some steps might be conditionally omitted based on the outcomes of previous steps. First, an 
image input device receives 10 a color image. The image is then converted 20 into a 
luminance-chrominance space. The input device can do this conversion or it can be done by 
another device to which the input signal is sent. High-pass filters are then applied 30 to the 
luminance and chrominance components. In embodiments, the high-pass filtered 
chrominance components are then combined 40 into a single component. The combined high- 
pass filtered chrominance component is then weighted. First, if the polarity of the combined 
high-pass-filtered chrominance component does not match that of the high-pass-filtered 
luminance component, the combined high-pass-filtered chrominance component is inverted 
60. Whether or not the combined high-pass-filtered chrominance component is inverted, the 
combined high pass filtered chrominance component is then multiplied 70 by a weight factor 
based upon the magnitude of the high-pass-filtered luminance component and, in 
embodiments, the magnitude of the high-pass-filtered chrominance component. The weight 
factor is smaller where the high-pass filtered luminance component is large, the weight factor 
is larger where the high-pass filtered luminance component is small. The weighted combined 
high-pass filtered chrominance component is then used 80 to modify the luminance 
component. The modified luminance signal is then sent to the output device, which outputs 
90 a grayscale image. 

[0030] Figure 5 schematically shows an apparatus for accomplishing the method 
disclosed herein. An image input device 1 10 would be the source of data corresponding to a 
color image. The image input device could be, for example, a color scanner, a personal 
computer or workstation with appropriate document or image creation software, a camera or a 
data storage device. If necessary, the color image data is then sent to a color transformation 
device 120 where it is converted to a luminance-chrominance representation. The color 
transformation device 120 can take various forms and operational details, such as, for 
example, software running on a processor, or an ASIC. It may also be a component of the 
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image input device, the color transformation device could be part of the image input device 
110. Next, the luminance and chrominance signals pass through an edge detector 130. The 
edge detector 130 uses high-pass filters to determine the luminance and chrominance edges in 
the original color image. In embodiments, the initial luminance signal and the high-pass 
filtered luminance and chrominance signals are then sent on to a processing unit 140. The 
processing unit 140 may be hardwired circuitry or software running on a processor. The 
processing unit 140 compares the polarity of the high-pass filtered luminance signal arid the 
high-pass filtered chrominance signal to determine an overall polarity correction for the high- 
pass-filtered chrominance signal. The processing unit may also weight the high-pass-filtered 
chrominance signal based upon the magnitude of the high-pass-filtered luminance signal. The 
luminance signal and the weighted high-pass-filtered chrominance signal are then sent to a 
feedback unit 150. The feedback unit 150 modifies the luminance signal based upon the 
weighted HPF chrominance signal. The modified luminance signal is then sent to grayscale 
output unit 160. Grayscale output unit 160 could be, for example, a printer, a display screen 
(such as, for example, on a telephone or PDA), facsimile machine, or electric paper. 

[0031] While the present invention has been described with reference to specific 
embodiments thereof, it will be understood that it is not intended to limit the invention to 
these embodiments. It is intended to encompass alternatives, modifications, and equivalents, 
including substantial equivalents, similar equivalents, and the like, as may be included within 
the spirit and scope of the invention. All patent applications, patents and other publications 
cited herein are incorporated by reference in their entirety. 
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